There is a pervasive belief in the MIS research community that PLS has advantages over other techniques when analyzing small sample sizes or data with non-normal distributions. Based on these beliefs, major MIS journals have published studies using PLS with sample sizes that would be deemed unacceptably small if used with other statistical techniques. We used Monte Carlo simulation more extensively than previous research to evaluate PLS, multiple regression, and LISREL in terms of accuracy and statistical power under varying conditions of sample size, normality of the data, number of indicators per construct, reliability of the indicators, and complexity of the research model. We found that PLS performed as effectively as the other techniques in detecting actual paths, and not falsely detecting non-existent paths. However, because PLS (like regression) apparently does not compensate for measurement error, PLS and regression were consistently less accurate than LISREL. When used with small sample sizes, PLS, like the other techniques, suffers from increased standard deviations, decreased statistical power,and reduced accuracy. All three techniques were remarkably robust against moderate departures from normality, and equally so. In total, we found that the similarities in results across the three techniques were much stronger than the differences.
In the Foreword to an MIS Quarterly Special Issue on PLS, the senior editors for the special issue noted that they rejected a number of papers because the authors attempted comparisons between results from PLS, multiple regression, and structural equation modeling (Marcoulides et al. 2009). They raised several issues they argued had to be taken into account to have legitimate comparison studies, supporting their position primarily by citing three authors: Dijkstra (1983), McDonald(1996), and Schneeweiss (1993). As researchers interested in conducting comparison studies, we read the Foreword carefully, but found it did not provide clear guidance on how to conduct "legitimate" comparisons. Nor did our reading of Dijksta, McDonald, and Schneeweiss raise any red flags about dangers in this kind of comparison research. We were concerned that instead of helping researchers to successfully engage in comparison research, the Foreword might end up discouraging that type of work, and might even be used incorrectly to reject legitimate comparison studies. This Issues and Opinions piece addresses the question of why one might conduct comparison studies, and gives an overview of the process of comparison research with a focus on what is required to make those comparisons legitimate. In addition, we explicitly address the issues raised by Marcoulides et al., to explore where they might (or might not) come into play when conducting or evaluating this type of study.
A significant amount of information systems (IS) research involves hypothesizing and testing for interaction effects. Chin et al. (2003) completed an extensive experiment using Monte Carlo simulation that compared two different techniques for detecting and estimating such interaction effects: partial least squares (PLS) with a product indicator approach versus multiple regression with summated indicators. By varying the number of indicators for each construct and the sample size, they concluded that PLS using product indicators was better (at providing higher and presumably more accurate path estimates) than multiple regression using summated indicators. Although we view the Chin et al. (2003) study as an important step in using Monte Carlo analysis to investigate such issues, we believe their results give a misleading picture of the efficacy of the product indicator approach with PLS. By expanding the scope of the investigation to include statistical power, and by replicating and then extending their work, we reach a different conclusion--that although PLS with the product indicator approach provides higher point estimates of interaction paths, it also produces wider confidence intervals, and thus provides less statistical power than multiple regression. This disadvantage increases with the number of indicators and (up to a point) with sample size. We explore the possibility that these surprising results can be explained by capitalization on chance. Regardless of the explanation, our analysis leads us to recommend that if sample size or statistical significance is a concern, regression or PLS with product of the sums should be used instead of PLS with product indicators for testing interaction effects.
Individual beliefs about technology use have been shown to have a profound impact on subsequent behaviors toward information technology (IT). This research note builds upon and extends prior research examining factors that influence key individual beliefs about technology use. It is argued that individuals form beliefs about their use of information technologies within a broad milieu of influences emanating from the individual, institutional, and social contexts in which they interact with IT. We examine the simultaneous effects of these three sets of influences on beliefs about usefulness and ease of use in the context of a contemporary technology targeted at autonomous knowledge workers. Our findings suggest that beliefs about technology use can be influenced by top management commitment to new technology and the individual factors of personal innovativeness and self-efficacy. Surprisingly, social influences from multiple sources exhibited no significant effects. Theoretical and practical implications are offered.